Wearable pupil-forming display apparatus

ABSTRACT

A wearable display apparatus has a headset for display from left-eye and right-eye near-eye catadioptric pupil-forming optical systems, each defining an exit pupil. Each optical system has an image generator to direct image-bearing light for a 2D image from an emissive surface. A curved reflective surface along the view axis is partially transmissive and defines a curved intermediate focal surface. A beam splitter reflects light toward the curved reflective surface. An image relay optically conjugates the formed image with a curved aerial image formed in air at the curved intermediate focal surface. The image relay has a prism having an input surface facing the emissive surface of the image generator, an output surface facing the curved intermediate focal surface, and a folding surface between input and output surfaces for folding the optical path for light generated by the image generator. An aperture stop for the relay is within the prism.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation-in-part of U.S. patentapplication Ser. No. 17/496,102, filed Oct. 7, 2021 and entitled“WEARABLE PUPIL-FORMING DISPLAY APPARATUS”, in the names of DavidKessler et al., which is a continuation of U.S. patent application Ser.No. 17/389,484 filed Jul. 30, 2021 and entitled “WEARABLE PUPIL-FORMINGDISPLAY APPARATUS”, in the names of David Kessler et al., which is acontinuation of U.S. patent application Ser. No. 17/139,167, filed Dec.31, 2020 and entitled “WEARABLE PUPIL-FORMING DISPLAY APPARATUS”, in thenames of David Kessler and Michael H. Freeman, and issued as U.S. Pat.No. 11,112,611, which, in turn, claims the benefit of U.S. Provisionalapplication Ser. No. 63/060,343, provisionally filed on Aug. 3, 2020entitled “WEARABLE PUPIL-FORMING DISPLAY APPARATUS” in the names ofDavid Kessler and Michael H. Freeman, incorporated herein by referencein its entirety. The present application is also related to commonlyassigned U.S. patent application Ser. No. 17/138,240 entitled “WEARABLEPUPIL-FORMING DISPLAY APPARATUS”, filed on Dec. 30, 2020 and to commonlyassigned U.S. patent application Ser. No. 17/133,912 entitled “SYSTEM,METHOD, AND NON-TRANSITORY COMPUTER-READABLE STORAGE MEDIA RELATEDWEARABLE PUPIL-FORMING DISPLAY APPARATUS WITH VARIABLE OPACITY ANDDYNAMIC FOCAL LENGTH ADJUSTMENT”, filed on Dec. 24, 2020 and issued asU.S. Pat. No. 11,137,610 on Oct. 5, 2021.

FIELD

The present disclosure generally relates to wearable display apparatusand more particularly to a wearable display device that providesaugmented reality (AR), mixed reality (MR), and extended reality (XR)viewing (collectively herein “AR”) with a catadioptric pupil-formingoptical system that renders a binocular 3D virtual image from a pair of2-dimensional (2D) displays.

BACKGROUND

Virtual image display has advantages for augmented reality (AR)presentation, including providing the capability for display of imagecontent using a compact optical system that can be mounted on eyeglassesor goggles, generally positioned very close to the eye (Near-EyeDisplay) and allowing see-through vision, not obstructing the view ofthe outside world. Among virtual image display solutions for AR viewingare catadioptric optics that employ a partially transmissive curvedmirror for directing image-bearing light to the viewer's eye and apartially reflective beam splitter for combining light generated at a 2Ddisplay with the real-world visible scene which forms a 3D image whenviewed binocularly.

Vision correction applications have employed wearable display devices inorder to enhance or compensate for loss of vision over portions of asubject's field of view (FOV). Support for these types of applicationscan require additional components and can introduce various factorsrelated to wearability and usability that contribute to the overallcomplexity of the optical design and packaging.

Among challenges that must be addressed with wearable AR devices isobtaining sufficient brightness of the virtual image. Many types of ARsystems, particularly those using pupil expansion, have reducedbrightness and power efficiency. Measured in NITS or candelas per squaremeter (Cd/m²), brightness for the augmented imaging channel must besufficient for visibility under some demanding conditions, such asvisible when overlaid against a bright outdoor scene. Other opticalshortcomings of typical AR display solutions include distortion, reducedsee-through transmission, small eye box, and angular field of view (FOV)constraints.

Some types of AR solution employ pupil expansion as a technique forenlarging the viewer eye-box. However, pupil expansion techniques tendto overfill the viewer pupil which wastes light, providing reducedbrightness, compromised resolution, and lower overall image quality.Pupil expansion techniques also typically have small field-of-viewattributes.

Challenging physical and dimensional constraints with wearable ARapparatus include limits on component size and positioning and, withmany types of optical systems, the practical requirement for folding theoptical path in order that the imaging system components beergonomically disposed, unobtrusive, and aesthetically acceptable inappearance. Among aesthetic aspects, compactness is desirable, withlarger horizontal than vertical dimensions.

Other practical considerations relate to positioning of the displaycomponents themselves. Organic Light-Emitting Diode (OLED) displays havea number of advantages for brightness and overall image quality, but cangenerate perceptible amounts of heat. For this reason, it is advisableto provide some distance and air space between an OLED display and theskin, particularly since it may be necessary to position these devicesnear the viewer's temples.

Still other considerations relate to differences between users of thewearable display, such as with respect to inter-pupil distance (IPD) andother variables related to the viewer's vision. Further, problemsrelated to conflict between vergence depth and accommodation have notbeen adequately understood or addressed in the art.

It has proved challenging to wearable display designers to provide theneeded image quality, while at the same time allowing the wearabledisplay device to be comfortable and aesthetically pleasing and to allowmaximum see-through visibility. In addition, the design of system opticsmust allow wearer comfort in social situations, without awkwardappearance that might discourage use in public. Providing suitablecomponent housing for wearable eyeglass display devices has proved to bea challenge, making some compromises necessary. As noted previously, inorder to meet ergonomic and other practical requirements, some foldingof the optical path along one or both vertical and horizontal axes maybe desirable.

SUMMARY

The Applicants address the problem of advancing the art of AR displayand addressing shortcomings of other proposed solutions, as outlinedpreviously in the background section.

The Applicants' solution uses pupil forming and can be distinguishedfrom pupil expansion systems known to those skilled in the art. Bycomparison with pupil expansion approaches, the Applicants' approachyields a more efficient optical system with improved image quality.Moreover, the eyes of the viewer can clearly see and be seen by others,with minimal impediment from the optics that provide the electronicallygenerated virtual image.

Wearable display apparatus of the present disclosure are well-adaptedfor systems that complement viewer capabilities, such as where a viewermay have visual constraints due to macular degeneration, low vision, orother condition of the eyes.

With these objects in mind, there is provided a wearable displayapparatus comprising a wearable display apparatus comprising a headsetthat is configured for display from a left-eye optical system and aright-eye optical system, wherein each optical system defines acorresponding exit pupil for a viewer along a view axis and comprises:

-   -   (a) an electroluminescent image generator that is energizable to        direct image-bearing light for a 2D image from an emissive        surface;    -   (b) a curved reflective surface disposed along the view axis and        partially transmissive, wherein the curved reflective surface        defines a curved intermediate focal surface;    -   (c) a beam splitter disposed along the view axis and oriented to        reflect light toward the curved reflective surface;    -   (d) an optical image relay that is configured to optically        conjugate the formed 2D image at the image generator with the        intermediate focal surface, wherein the optical image relay        comprises:    -   (i) a prism having an input surface facing toward the emissive        surface of the image generator, an output surface facing toward        the intermediate focal plane, and a folding surface extending        between the input and output surfaces and configured for folding        an optical path for light generated by the image generator,        wherein an aperture stop for the relay lies within the prism;    -   (ii) at least a first plano-aspheric lens in optical contact        against the prism input surface and configured to guide the        image-bearing light from the image generator toward the folding        surface;    -   wherein the relay, curved mirror, and beam splitter are        configured to form the exit pupil for viewing the generated 2D        image superimposed on a portion of a visible object scene,    -   wherein combined images from both left- and right-eye optical        systems form a 3D image for the viewer;    -   and    -   (e) a plurality of sensors coupled to the headset and configured        to acquire measured data relating to the viewer.

According to an embodiment of the present disclosure, there is provideda wearable display apparatus comprising a headset that is configured fordisplay from:

(a) a left-eye optical system;

(b) a right-eye optical system,

wherein each optical system is a near-eye catadioptric pupil-formingoptical system that defines a corresponding exit pupil for a vieweralong a view axis and comprises:

-   -   (i) an electroluminescent image generator that is energizable to        direct image-bearing light for a 2D image from an emissive        surface along an optical path;    -   (ii) a curved reflective surface disposed along the view axis        and partially transmissive, wherein the curved reflective        surface defines a curved intermediate focal surface;    -   (iii) a beam splitter disposed along the view axis and oriented        to reflect light toward the curved reflective surface;    -   (iv) an optical image relay that is configured to optically        conjugate the formed 2D image at the image generator with a        curved aerial image formed in air at the curved intermediate        focal surface, wherein the optical image relay comprises:        -   a prism having an input surface facing toward the emissive            surface of the image generator, an output surface facing            toward the curved intermediate focal surface, and a folding            surface extending between the input and output surfaces and            configured for folding the optical path for light generated            by the image generator, wherein an aperture stop for the            relay is formed within the prism;        -   wherein the relay, curved mirror, and beam splitter are            configured to form the exit pupil for viewing the generated            2D image superimposed on a portion of a visible object            scene,            wherein combined images from both left- and right-eye            optical systems form a 3D image for the viewer;            (c) a stationary circuit board that houses a first set of            electronic components having a fixed position within the            headset;            (d) a movable circuit board that is mechanically coupled to            a transport apparatus that is configured to impart movement            to either the left-eye or right-eye optical system and that            houses a second set of electronic components,            wherein a flexible connection maintains signal communication            between the movable circuit board and the first set of            electronic components on the stationary circuit board; and            (e) an actuator that is energizable to move the transport            apparatus away from or toward the stationary circuit board            according to viewer monitoring.

DRAWINGS

FIG. 1A is a schematic front view showing placement of opticalcomponents of the system.

FIG. 1B is a schematic side view showing placement of optical componentsof the system.

FIG. 2A is a schematic front view showing the optical path throughcomponents of the system.

FIG. 2B is a schematic side view showing the optical path throughcomponents of the system.

FIG. 3A is a schematic that shows, in perspective view, components of anoptical apparatus for AR viewing.

FIG. 3B is a simplified schematic of FIG. 3A.

FIG. 4 is a schematic that shows, from an alternate perspective view,components of an optical apparatus for AR viewing.

FIG. 5 is a side view schematic of an image relay.

FIGS. 6A, 6B are schematic, showing the image relay and components forforming the exit pupil.

FIG. 6C is a schematic view that shows an alternate embodiment of thepresent disclosure that employs the imaging path for eye tracking.

FIGS. 7A-7C show various features of an embodiment useful forcompensating for macular degeneration.

FIG. 8 is a block diagram illustration showing integrated components foran HMD system in an exemplary embodiment.

FIG. 9 is a perspective view that shows positions of various sensors,processor, and components of the head-mounted display according to anembodiment.

FIG. 10 is a schematic diagram that shows components of an interpupildistance adjustment system for the head-mounted display.

FIG. 11A is a schematic diagram showing optical relay components usedfor focal plane adjustment.

FIG. 11B is a perspective view that shows position of an actuatorrelative to the display and to corresponding optics.

FIG. 11C is a perspective view from behind the display component,showing a piezoelectric actuator mounted to a plate behind the display.

FIG. 11D is a side view showing components of a dynamic focus adjustmentapparatus.

FIG. 12 is a schematic diagram showing visual accommodation andvergence-accommodation conflict.

FIG. 13 is a schematic diagram that shows aspects of dithering geometryfor enhanced image resolution.

FIG. 14 is a perspective view that shows a wearable display headset thatis configured to maintain IPD adjustment suited to the individualviewer.

FIG. 15 shows portions of the headset removed from the viewer forehead.

FIG. 16 shows internal components of the headset with an external coverremoved.

FIG. 17 shows internal components of the headset with additionalcomponents removed.

FIGS. 18A, 18B, and 18C show a basic form of flexible circuit as a typeof ribbon cable having printed circuit traces as well as connectors.

FIG. 19 shows a perspective view of an embodiment in which only theright-eye optical system is shifted; the left-eye optical system isstationary with respect to the frame.

FIG. 20 is a perspective view that shows headset optics configured forIPD adjustment according to an embodiment of the present disclosure.

FIG. 21A shows a front view of the headset optics of FIG. 20 with aminimum IPD setting.

FIG. 21B shows a front view of the headset optics of FIG. 20 with amaximum IPD setting.

FIG. 22 is an exploded view showing the relative position of an IPDadjustment apparatus with respect to headset optics.

FIGS. 23A and 23B show positions of a cam apparatus for IPD adjustment.

FIGS. 24A and 24B show minimum and maximum positions for IPD adjustmentaccording to an alternate embodiment of the present disclosure.

DESCRIPTION

The following is a detailed description of the preferred embodiments ofthe disclosure, reference being made to the drawings in which the samereference numerals identify the same elements of structure in each ofthe several figures.

While the devices and methods have been described with a certain degreeof particularity, it is to be noted that many modifications may be madein the details of the construction and the arrangement of the devicesand components without departing from the spirit and scope of thisdisclosure. It is understood that the devices and methods are notlimited to the embodiments set forth herein for purposes ofexemplification. It will be apparent to one having ordinary skill in theart that the specific detail need not be employed to practice accordingto the present disclosure. In other instances, well-known materials ormethods have not been described in detail in order to avoid obscuringthe present disclosure.

Reference throughout this specification to “one embodiment,” “anembodiment,” “one example,” or “an example” means that a particularfeature, structure, or characteristic described in connection with theembodiment or example is included in at least one embodiment of thepresent disclosure. Thus, appearances of the phrases “in oneembodiment,” “in an embodiment,” “one example,” or “an example” invarious places throughout this specification are not necessarily allreferring to the same embodiment or example. Furthermore, the particularfeatures, structures, or characteristics may be combined in any suitablecombinations and/or sub-combinations in one or more embodiments orexamples.

In the context of the present disclosure, the term “eyebox” has itsconventional meaning in the HMD arts, as functionally equivalent to “eyemotion box” and similar phrases. The eyebox is that volume of spacewithin which the viewable image is formed by an optical system or visualdisplay. When the viewer's pupil is within this volume, the viewer cansee all of the generated display content; with the pupil is outside ofthis volume, the user is typically not able to view at least someportion of the display content. A larger eyebox is generally desirable,as this allows for lateral and axial movement of the eye, while stillmaintaining a full field of view. The size of the eyebox relatesdirectly to the size of the exit pupil for a display system.

Several (or different) elements discussed herein and/or claimed aredescribed as being “coupled,” “in communication with,” “integrated,” or“configured to be in communication with” or a “system” or “subsystem”thereof. This terminology is intended to be non-limiting and, whereappropriate, be interpreted to include, without limitation, wired andwireless communication using any one or a plurality of a suitableprotocols, as well as communication methods that are constantlymaintained, are made on a periodic basis, and/or made or initiated on anas-needed basis.

Where they are used, the terms “first”, “second”, and so on, do notnecessarily denote any ordinal, sequential, or priority relation, butare simply used to more clearly distinguish one element or set ofelements from another, unless specified otherwise.

In the context of the present disclosure, the term “coupled” is intendedto indicate a mechanical association, connection, relation, or linking,between two or more components, such that the disposition of onecomponent affects the spatial disposition of a component to which it iscoupled. For mechanical coupling, two components need not be in directcontact, but can be linked through one or more intermediary components.

An embodiment of the present disclosure provides AR viewing and displayhaving a large FOV with an optical system having an optical path thatfolds in the horizontal or x-direction, the direction substantiallyparallel (+/−15 degrees) to a line between left and right pupils of aviewer, for forming an intermediate image to the curved mirror. Anembodiment of the AR system of the present disclosure has a componentarrangement as shown schematically in the front view of FIG. 1A and fromthe side in FIG. 1B. The corresponding light path is shown schematicallyin FIGS. 2A and 2B, respectively. A flat-panel display is energized asan image generator 10 to form an image and to direct image-bearing lightthrough beam-shaping optics and to a folding prism 20 that redirects theimage-bearing light towards a beam splitter 24 and to a curved mirror 30for forming the virtual image from electronically generated imagecontent. Image generator 10 can be a display that emits light, such asan organic light-emitting device (OLED) array or a liquid crystal arrayor a micro-LED array with accompanying lenslets, or some other type ofspatial light modulator useful for image generation.

In order to address the need for improved overall imaging performance,wider FOV, compactness, and other factors outlined in the background,embodiments of the present disclosure have a number of features shownparticularly in FIGS. 3A, 3B, and 4. Specific features of interestinclude:

-   -   (i) relay of the image generator 10 to form a curved        intermediate image I as a conjugate image. As a type of “aerial”        image, intermediate image I is formed in air, serving as the        optical “object” for forming the virtual image. Intermediate        image I is formed along the curved focal surface of curved        mirror 30, with the approximate position shown by a dashed line        in FIG. 5. An optical relay 40, with particular structure as        described in more detail subsequently, conjugates the image        formed from image generator 10 to the curved intermediate image        I along the focal surface. Curved mirror 30 is partially        transmissive, such as between about 30% to 70% transmissive, for        example, allowing visibility of the real-world object scene to        the viewer. A nominal transmission range of about 50% is useful        in many applications.    -   (ii) use of a cylindrically curved quarter-wave plate (QWP)        between mirror 30 and beam splitter 24. Curvature of this        element helps to reduce variations of the retardation imparted        to the image-bearing light by the QWP over the field of view.    -   (iii) large exit pupil 44. System optics can form a 10 mm exit        pupil at the viewer's eye-box for eye E. Forming a suitably        sized pupil for the viewer helps to provide an eyebox of        reasonable dimensions to allow eye movement, without noticeable        vignetting. Also, an enlarged eyebox permits the headset to move        or slip without noticeable degradation of the viewed image(s).        The apparatus does not need to provide pupil expansion, such as        is used in existing wearable display apparatus, but uses        pupil-forming optics for improved efficiency and brightness, as        well as for improved image resolution.

Significantly, the eyes of the viewer can clearly see and be seen byothers, with minimal impediment from the beam splitter and curved mirroroptics that provide the electronically generated virtual image.

With the optical arrangement shown, the aperture stop AS lies withinprism 20 of the image relay, along or very near the fold surface that isprovided. This arrangement is advantageous for component packaging andspacing, allowing prism 20 to be reduced in size over otherconfigurations using a folding prism.

The given design allows an FOV along the horizontal (x) axis, the axisparallel to a line between left and right pupils of the viewer's eyes,of greater than 50 degrees. The FOV aspect ratio (horizontal:vertical)equals or exceeds 1.5. Digital correction is not needed for distortionor lateral color.

According to an embodiment, curved mirror 30 has a conic surface shape.The conic shape is advantaged, in the embodiment shown herein, helpingto control chief ray angles, thus correcting for distortion.

Depending on whether or not polarization is used for configuring lightpaths, beam splitter 24 can be either a polarization-neutral beamsplitter or a polarization beam splitter. Beam splitter 24 can be, forexample, a wire grid polarization beam splitter as shown in FIG. 4.

Image Relay 40

FIG. 5 shows an enlarged side view of relay 40 for relay of the displayimage from an electroluminescent display (such as an OLED in theexamples shown) to the focal surface position of mirror 30 (shown in aperspective view in FIG. 4) and for shaping the relayed image tosuitable curvature to correct distortion. A concave-plano field lens L1,with sides truncated along the vertical direction as shown in FIG. 4 inorder to reduce weight and provide a more compact system, shapes thelight from OLED display image generator 10, providing a beam to ameniscus singlet lens L2.

From lens L2, the imaging light goes to a doublet L3 having aconcave/convex flint glass lens cemented to a crown glass lens.

An aspheric plano-convex lens L4 is in optical contact with the inputface of prism 20, such as cemented to prism 20. A second plano-asphericlens L5 can be cemented to the output face of prism 20. This cementedarrangement facilitates alignment of these optical components. Accordingto an alternate embodiment, only a single plano-aspheric lens L4 isdeployed at the prism 20 input surface.

The turning surface 26 of prism 20 is coated to enhance reflection.Hypotenuse or turning surface 26 of the prism is essentially the relay(and system) aperture stop.

Intermediate image I is formed in the shape and location of the focalsurface of the curved mirror. Proceeding backward along the optical pathfrom intermediate image I are the following components:

-   -   Plano-asphere lens L5;    -   Folding prism 20 with turning surface 26;    -   plano-asphere lens L4;    -   Doublet L3;    -   Meniscus singlet L2;    -   Field lens L1;    -   Image source or generator, display 10

FIG. 6A and simplified FIG. 6B show an alternate view of the displayoptics from exit pupil 44. Chief rays are shown in FIG. 6B; these chiefrays converge at the position of exit pupil 44 at eye E. FIG. 6B alsoshows the approximate position of an intermediate pupil Pi at theaperture stop, near the folding surface 26 of prism 20.

As shown in FIGS. 1A and 4, the image generator is disposed to directimage-bearing light beam in a horizontal direction and along a path thatlies above eye-level, as the display optics are normally worn by asitting or standing viewer. Prism 20 can be tilted slightly away fromthe forehead of the viewer, to direct light in front of the face at anoblique angle to vertical, as shown in the embodiment of FIG. 2B.

The layout and routing of the optical path are particularly suitable forproviding augmented reality 2D and 3D viewing in a wearable device.Using relay 40 allows the positioning of image generator 10 to be out ofthe direct field of view; in addition, relay 40 allows image generator10 to be positioned at sufficient distance away from the skin surface toavoid contact and consequent discomfort. The use of a first x-direction(horizontal) fold, followed by a y-direction (vertical) folding enablesthe imaging optics to be compactly packaged with reasonable optical pathdistance to allow a measure of light beam shaping and correction. Prism20 can be rotated over at least a range of angles about the x axis,allowing a measure of alignment as well as adaptation to differentmirror 30 curvatures. Employing a curved surface for an optional QWPcomponent helps to reduce variations, over the FOV, of retardationimparted by the QWP; excessive variation over the field may otherwisecause some brightness fall-off.

Using a wire-grid polarizer reduces light loss, allowing high levels ofvisibility to the external, real-world object scene content, along withreduced light leakage over other polarization components.

Image source 10 may be unpolarized. In one embodiment, a polarizing beamsplitter is used, such as a wire grid splitter made by Moxtek, Inc.,Orem, Utah. This type of beam splitter reflects only one polarization,usually S polarization, towards the conic combiner. The orthogonalpolarization, P polarization, is transmitted and is absorbed (absorbernot shown). To prevent the small amount of P light from being reflected,an optional polarizer can be placed at the image source.

The mirror 30 provides a conic combiner in embodiments shown, with poweronly for the generated image and not for the visible field. The curvedmirror 30 can be a double conic for improved image formation. Varioustypes of coatings can be provided on the mirror 30 combiner, including,but not limited to dichroic coatings, metal coatings, such as to providea half-silvered reflector, electrochromatic coatings, anti-reflection(AR) coatings. Mirror 30 can be fully or partially reflective or fullyor partially transparent, with some amount of reflectivity.

Embodiments of the present disclosure provide a measure of distancebetween the image generator (OLED or other spatial light modulatordevice) and the face and temples of the viewer. This helps to preventdiscomfort due to heat where the wearable display is worn for anextended period of time.

The particular arrangement of image-forming components provides suitableimage quality and high resolution to allow reading and other visualactivity involving fine detail.

According to an embodiment of the present disclosure, the optical systemdescribed herein is suitable for applications requiring sensitivity tothe viewer, including not only viewer comfort, but some level of visionmonitoring and adaptation. For example, the apparatus described hereincan be used as part of a system for compensating for vision problems. Byway of example, FIGS. 6C and 7A-7C show various features of anembodiment useful for compensating for macular degeneration. This typeof application can require a measure of viewer monitoring andadaptation, possibly including adjustment of generated data contentsuitable for the viewer.

Gaze Tracking

According to an aspect of the present disclosure, gaze tracking can beprovided as part of the wearable optics system and used to adjust systemparameters according to perceived focus of attention for the viewer.Cameras and infrared (IR) light sources provided on a headset, as shownsubsequently, can provide the gaze-tracking function and correspondingangular measurement data. Gaze tracking can be combined with thecontroller and with a camera image FOV intake. For instance, change ofthe image aspect ratio for generated image data content may beappropriate, allowing the system to adapt image content to thedimensional parameters available from the image generation system. Thus,for example, cameras associated with the HMD can oversample thereal-world input from the object scene, acquiring a wider FOV than canbe displayed by system optics. Gaze tracking identifies the actual FOVavailable to the viewer. The resulting buffered images are related tothe reduced FOV video that can be generated, as controlled by using thesector of the FOV identified using eye gaze recognition.

FIG. 6C is a schematic view that shows an alternate embodiment of thepresent disclosure that employs the imaging path itself for eye trackingand provides 1:1 imaging of the viewer's iris. IR light, or othersensing light, is directed along the optical path, such as generatedfrom, through, or at some other point along the output path of, imagegenerator display 10. Folding surface 26 can be formed as a dichroicsurface, treated to direct the sensing light to beam splitter 24 and tocurved mirror 30 and to the iris of eye E. Sensor light returning fromthe iris generally retraces the light path to prism 20. A portion ofthis returned light from the viewer's eye, as shown by a dashed line inFIG. 6C, can be transmitted, rather than reflected, through surface 26and be conveyed through a complementary facing prism 60 to a trackingsensor 62, which can be a camera, imaging array, or other imaging sensoraccording to an embodiment. This returned light sensed at sensor 62 canbe IR light, for example. The IR light source can be directed to the eyefrom any suitable position, and can be directed through or alongsidecomponents on the optical axis. Surface 26 of the prism is thenconfigured to reflect most of the visible light, performing its turningfunction for image-bearing light, and to transmit IR light suitably forgaze tracking.

Other configurations are possible. Thus, for example, a dichroic coatingcan be employed for surface 26, or some other coating can be employedthat provides the needed redirection by reflection of the bulk ofimage-bearing light, while also allowing sufficient light leakage forsensing.

The embodiment of FIG. 6C can be modified in a number of ways to alloweye gaze tracking through portions of the imaging path. For example,with corresponding changes to components, the use of IR light as asensed light can be replaced by employing visible light. This wouldrequire surface 26 to be partially reflective in the visible range, suchas 90% reflective and 10% transmissive, for example, wherein thetransmitted light is the sensed portion. One or more additional lenses,not shown in FIG. 6C, could be provided in the path of light to sensor62 for gaze tracking using the image path as described herein.

Headset Configurations

FIGS. 7A-7C show a stereoscopic display headset 100 that has separateoptical paths for the left and right eyes of a viewer. As particularlyshown in FIGS. 7A and 7C, the optical relay 40 can be compactly packagedas part of a headset 100 within an optical module 42 that is disposedabove eye level, as the system is normally worn. Beam splitter 24 andcurved mirror 30 are arranged to lie along the visual axis of theviewer. An adjustable strap 102 can be provided to allow adaptation toviewer anatomy. A control logic processor 34 can include the neededelectronics for controlling operation of the optical apparatus;processor 34 components can be mounted above the visual axis anddisposed away from the viewer's forehead. Control logic processor 34 canalternately be separated from, but in wired or wireless signalcommunication with, headset 100.

Eye-tracking can be provided from the headset using the arrangementpreviously described with respect to FIG. 6C or using one or moreeye-tracking cameras 46, working in conjunction with illumination LEDs48, typically infra-red (IR) sources, as shown in FIG. 7B.

FIG. 8 shows a headset 100 that incorporates the optical systemdescribed herein as part of a wearable system for providing userinformation and guidance in performing a task or assignment. Opticalrelay 40 and its associated components are packaged within an opticalmodule 42. A variety of sensors 80, described in detail subsequently,can include one or more SLAM sensors 82, as well as sensors for variousenvironmental or ambient conditions such as temperature, humidity, andthe like, viewer-related, contextual data, cameras, and otherinformation-gathering devices can be integrated with or in wireless orwired signal communication with control logic and related processingcomponents of the HMD headset. Sensors can be provided to support ARimaging, VR imaging, or mixed AR/VR imaging modes of operation.

Connection to power and to signal sources for the headset can beobtained by connection of the headband of headset 100 with externalpower and signal sources, such as other computing and processingequipment worn or carried by the viewer.

System Block Diagram

FIG. 8 is a diagrammatic block diagram illustration showing, for anexemplary embodiment, interrelation of integrated components for an HMDsystem including input sensors 80, output components and systems 90,logic and control components including processors, graphic processingunits (GPU), MVC and other devices. Sensors 80 can includehigh-resolution cameras, multiple displays per eye, 6 to 9 degrees offreedom sensor or other sensors necessary for detection ofhand-gesturing, head-gesturing, voice control, positional location, andestimation or navigation, as well as optical character recognition(OCR), tracking, marker-based or markerless-based AR, location, SLAMsensors, concurrent odometry and mapping sensors, microphones andnoise-cancelling microphones, and any other sensors which could becoupled to and used on an AR/VR headset. Previous figures, for example,gave a diagrammatic illustration of a placement of an IR light for theeye-tracking subsystem. The perspective view of FIG. 9 shows positionsof various sensors, processor, and components coupled to headset 100according to an embodiment of the present disclosure.

Among other sensor technologies which may be housed on the HMD aremanual control inputs. These can include digital buttons, which mayinclude power buttons, and a D-Pad or control-pad for accessing andcontrolling functions by the user, which may or may not be in a dongle;and if not in a dongle then it may exist on the headset or in a wired orwireless remote control. The sensors listed above may include theiroperating systems and output. The control mechanism may also respond toother types of input, including voice command, SLAM, eye tracking, heador hand gesturing, or any other method which can be employed with thesensors and systems mentioned above.

HMD 100 may also house connectors such as power connection forrecharging a battery or for direct connection to an AC source, for theHMD as well as for related input and output devices. There can also beadditional external connectors for HDMI, sound, and other input/outputs,such as additional image overlay display, or for a diagnostics protocolfor upgrading the system. The HMD may also house the microprocessor(s)control circuits. HMD 100 may also contain one or more display per eye,allowing the use of any number of additional projectors, like Picoprojectors, or micro-displays. The displays may be used to projectthough either catoptric system, a dioptric system, or catadioptricsystem, or combinations thereof, such as to generate anultra-short-throw image onto reflective lenses or to project to someother surface, which can be clear plastic, like a polycarbonate resinthermoplastic (Lexan).

The HMD 100 may also house a rechargeable battery which is not typicallyremoved, thus, providing spare energy to continue to power the HMD whenthe removable battery is exhausted or removed. While this battery may besmaller and only have a run-time of several minutes, it can provide theHMD with a “hot-swap” battery system that permits a user to keep viewingfrom the HMD for a time after the removeable battery has died or beenremoved.

HMD 100 may also include a strap and counterweight or other headgear tobalance the HMD and maintain its position on the head. The HMD maycontain a “pinch adjustor” to adjust strap 102. In addition, the HMD mayor may not include a “dongle” whereby one or more of the systems orsubsystems may be connected via wired or wireless to another device,such as could be worn on a belt or carried in a pocket to reduce theoverall weight of the HMD 100. In one embodiment, the HMD may beconnected to another device which is providing power, while in analternative embodiment, the HMD may have its own power from the mains orfrom wireless power transmission or from a battery.

Further, in another embodiment, the HMD may house other subsystems suchas the cameras, the microcontrollers, the connectors, central processingunit, graphics processing unit, software, firmware, microphones,speakers, display, and collector lens; the displays, and othersubsystems.

In another embodiment, the HMD may contain a front facing sensor arrayalong with other sensors mentioned above and optical characterrecognition (OCR) and/or cameras to read and/or measure information fromthe real world object scene. Additionally, the HMD may contain one ormore connectors to connect via wire to the outside world for power anddata (i.e. USB, HDMI, MiniUSB).

Alternatively, some parts of the system mentioned herein may be in adongle attached to the HMD via wire or wireless connection.Alternatively, some portions of the system mentioned herein may becontained in a connected device, like a laptop, smart phone, or Wi-Firouter. Alternatively, some parts of the system mentioned herein may becontained in a smartphone or may be transferred back and forth from asmartphone to the HMD, when synced, such as the HMD displaying thesmartphone apps and other features of the smartphone that wouldotherwise be displayed on the smartphone display. Alternatively, the HMDmay contain and display all the features that a smartphone can.

In another aspect of the present disclosure, HMD 100 may contain all thefeatures of a typical smartphone and no connection may be needed with asmartphone to have all the smartphone features, like web or cellcalling, app use, SMS, MMS, or similar texting, emailing, logging on tothe internet, and the like.

According to an aspect of the present disclosure, the HMD headset mayprovide a computer mediated video shown on the reflective lens layersuch that the wearer may see both the real world and the augmented videoat the same time. In this aspect of the disclosure, such features asvoice/speech recognition, gesture recognition, obstacle avoidance, anaccelerometer, a magnetometer, gyroscope, GPS, spatial mapping (as usedin simultaneous localization and mapping (SLAM)), cellular radiofrequencies, Wi-Fi frequencies, Bluetooth and Bluetooth Lightconnections, infrared cameras, and other light, sound, movement, andtemperature sensors may be employed, as well as infrared lighting, andeye-tracking.

Batteries and other power connections may be needed for various devices,but are omitted from schematic figures for clarity of other features.

SLAM Sensors

Embodiments of the disclosure can further include mechanisms and logicthat provide SLAM (simultaneous localization and mapping) capabilitiesto support the viewer. SLAM uses statistical techniques to map theviewer's environment and to maintain information on the viewer'srelative position within that environment. For example, an image from aSimultaneous Localization and Mapping (SLAM) camera configured for thewearable unit can detect a location of the HMD wearer within the givenenvironment. SLAM capabilities can also be useful where some portion ofthe viewer's FOV is blocked or otherwise obscured. Using SLAM allows thesystem to present portions of the real-world object scene in theelectronically generated image. SLAM capability allows generation anddisplay of image content related to the real-world viewer environment.

To support SLAM capabilities, a headset as shown in the example of FIG.9 can include one or more SLAM sensors 82, such as cameras. In somecases, the SLAM camera may include a visual spectrum camera, an infrared(IR) camera, or a near-IR (NIR) camera. Additionally, or alternatively,the SLAM camera may be include or exclude the viewer from its field ofview. Techniques for providing SLAM capabilities are known to thoseskilled in the mapping arts and can include processing of local imagecontent as well as use of tracking data.

SLAM logic can be provided by control logic processor 34 or by anexternal processor that is in signal communication with processor 34,including processors that are connected to the wearable display deviceby a wired connection or, alternately, processors that are in wirelesscommunication with control processor 34.

IPD Adjustment System

Embodiments of the present disclosure address the need for HMD systemadjustment to viewer anatomy, with benefits for system efficiency andusability. One aspect of variable viewer anatomy relates to inter-pupildistance (IPD). This well-known characteristic relates to overall headdimensions and position of the eye sockets. Mismatch of IPD by thedevice can make it difficult to provide image content and neededfunctions and can make the HMD difficult to wear and use for someviewers. Some systems provide manual methods for IPD adjustment. Asshown in the perspective view of FIG. 10, the Applicant's HMD canprovide an automated IPD adjustment system 110 that measures the IPD forthe viewer and responds by adjusting the inter-pupil spacing, changingthe distance between the left and right optical modules 42 l and 42 r,respectively.

Referring to the FIG. 10 depiction, IPD adjustment system 110 can havean actuator 112 that urges one or both left and right optical modules 42l, 42 r either farther apart, as shown in distance IPD2, or closertogether as in distance IPD1. This adjustment can be performed atpower-up or other system activation, at operator command, or atpredetermined periods, or at some other time or in response to otherevent. According to an embodiment of the present disclosure, IPDadjustment settings can be stored for each viewer who has previouslyused the HMD. Then, upon entry of information identifying the viewer,the stored IPD adjustment settings can be restored as presets.

There may even be an external device, motorized or mechanical, whicheffects the activation of the preset IPD upon instruction prior towearing the HMD.

In one embodiment, to execute the IPD adjustment function, eye-trackingcameras 46 on both left and right optical modules 42 l, 42 r obtainimage content that allows pupil center detection by processor 34 logic.This logic determines the relative location of actual pupil centers andIP distance and determines whether or not the IPD between pupil centersis compatible with the positioning of left and right optical modules 42l, 42 r. If positioning is appropriate, no IPD adjustment is necessary.Otherwise, an actuator 112 can be energized to translate one or bothleft and right optical modules 42 l, 42 r in the horizontal orx-direction as shown in FIG. 10. At various incremental positions,feedback logic can again measure and calculate any needed adjustmentuntil a suitable IPD is achieved. For IPD adjustment, at least one ofthe left and right optical modules 42 l and 42 r can be movable; theother module 42 l or 42 r can also be movable or may be stationary.

Adjustable Focus

As a useful default for most virtual image viewability, HMD optics aretypically designed to form the virtual image so that it appears to be atoptical infinity, that is, with rays substantially in parallel to theoptical axis. In an embodiment, the HMD optics are designed to providethe augmented virtual image at about 1 m distance from the viewereyebox, equivalent to approximately 1 diopter prescription glasses.

In some applications, closer focal length is advantageous. To achievethis, the Applicant's solution provides measurement and adjustment fordiopter adjustment of the optical relay 40 optics. Referring to FIG.11A, there is shown a schematic for relay 40 components that includes anactuator 122 and associated components as part of a focal planeadjustment system 120. By changing image generator 10 position along anaxis A, a change in focal position is effected, such as with imagegenerator 10 shifted to the dashed outline position denoted for imagegenerator 10′ in FIG. 11A. This movement causes a corresponding shift ofintermediate image I to the position shown as image I′ in relay 40. Thesplitter 24 and combiner, curved mirror 30, then condition theimage-bearing light to provide a virtual image at a shifted spatiallocation. Actuator 122 can be a linear piezoelectric actuator, forexample, capable of high-speed change between positions. One or moreactuators 122 can be used for moving any of the components describedhereinabove with relation to optical relay 40 in order to adjust theposition of the focal plane.

FIGS. 11B, 11C, and 11D show different views of relay optics with focalplane adjustment system 120. FIG. 11B is a perspective view that showsposition of actuator 122 relative to image generator 10 andcorresponding optics. FIG. 11C is a perspective view from behind imagegenerator 10, showing a piezoelectric actuator 122 mounted to astationary plate 124 behind image generator 10 (not visible in the viewof FIG. 11C). Actuator 122, such as a piezoelectric actuator, translatesimage generator 10 along axis A, which extends orthogonally to theelectroluminescent display component surface of image generator 10. FIG.11D shows a side view of adjustment system 120 components. One or morestabilizing posts 126 has a screw and a compression spring 162 formaintaining image generator 10 in position along the optical path, sothat movement of image generator 10 is constrained to the axialdirection (axis A) during actuation.

One difficulty with the change in focal length relates tovergence-accommodation conflict, as shown in the schematic diagram ofFIG. 12. Vergence-accommodation conflict (VAC) is a vision phenomenonthat is familiar to developers of three-dimensional (3D) displays andvirtual-reality (VR) display devices. Under real-world normal viewingconditions, where only the object scene is in view as shown in part (a)of FIG. 12, the viewer's eyes converge, rotating toward one another tofocus on closer objects and, correspondingly, away from one another tofocus on objects at further distance. Accommodation, as the processwhere the lenses of the eyes focus on a close or far away object, isconsistent with convergence for normal viewing in the part (a) depictionand involves epipolar geometry of the HMD cameras and the wearer's eyes.Thus, accommodation and convergence can be considered to be coupled.

Focal plane adjustment system 120 (FIGS. 11A-D) also employseye-tracking sensors and related data for determining when there is adiscrepancy between focal planes for the object world FOV and thegenerated image content. The focal plane position for the generatedimage can be computed according to system optical geometry for thecomponents defined hereinabove. Later adjustments to the geometry,executed by system logic for shifting focal plane position, can berecorded and used to recalculate focal plane position.

In FIG. 12 part (a), the viewer sees a real object, i.e., the viewer'seyes are verged on the real object and gaze lines from the viewer's eyesintersect at the real object. As the real object moves nearer the user,as indicated by the arrow, each eye rotates inward (i.e., converges) tostay verged on the real object. As the real object gets closer, the eye“accommodates” to the closer distance, through reflexive, automaticmuscle movements, by adjusting the eye's lens to reduce the focallength. In this way, accommodation adjustment is achieved. Thus, undernormal viewing conditions in the real world, the vergence distance(d_(v)) equals the accommodation distance (d_(f)).

In 3D displays and VR systems, however, these two processes can bedecoupled, as shown in part (b) of FIG. 12. FIG. 12 shows an exampleconflict between vergence and accommodation that can occur withstereoscopic three-dimensional displays, in accordance with one or moreembodiments. In this example, an observer is looking at the virtualobject displayed on a 3D electronic display; however, the observer'seyes are verged on and gaze lines mapped from the observer's eyesintersect at the virtual object, which is at a greater distance from theobserver's eyes than the image formed by the 3D electronic display. Asthe virtual object from the 3D electronic display is rendered to appearcloser to the viewer, each eye again rotates inward to remain verged onthe virtual object, but the focus distance of the image is not reduced;hence, the observer's eyes do not accommodate, as in part (a). Thus,instead of increasing the optical power to accommodate for the closervergence depth, the eye maintains accommodation at a display distanceassociated with the 3D electronic display. Thus, the vergence depth (dv)often does not equal the focal length (df) for the human eye for objectsdisplayed from 3D electronic displays. This discrepancy between vergencedepth and focal length is referred to as “vergence-accommodationconflict” or VAC. A user experiencing only vergence or onlyaccommodation, and not both vergence and accommodation, can eventuallyexperience some degree of fatigue, dizziness, discomfort,disorientation, and even nausea in some cases.

In order to compensate and correct VAC, the FIG. 11A-D relay optics canadjust the relative position of intermediate image I to I′, moving thevirtual image focal plane more closely toward the real-world focalplane.

Model Controller

Embodiments in accordance with the present disclosure may be provided asan apparatus, method, computer program, hardware/software, statemachine, firmware, and/or product. All of the systems and subsystems mayexist, or portions of the systems and subsystems may exist to form theapparatus described in the present disclosure. Accordingly, one or moreportions of the Applicant's solution may take the form of an entirely orpartial hardware embodiment, a predominantly software embodiment(including firmware, resident software, micro-code, etc.), or anembodiment combining software and hardware aspects in some combinationthat may all generally be referred to herein, without limitation, as a“unit,” “module,” or “system.” Furthermore, one or more portions orfunctions for the present disclosure may take the form of a computerprogram product or products embodied in any tangible media of expressionor storage having computer-usable program code embodied in or otherwiserepresented using the media. Any combination of one or morecomputer-usable or computer-readable media (or medium) may be utilized,including networked combinations that utilize remote processingcomponents. For example, a random-access memory (RAM) device, aread-only memory (ROM) device, an erasable programmable read-only memory(EPROM or Flash memory) device, a portable compact disc read-only memory(CDROM), an optical storage device, and a magnetic storage device.Computer program code for carrying out operations of the presentdisclosure may be written in any combination of one or more programminglanguages. Further, the intelligence in the main circuitry may besoftware, firmware, or hardware, and can be microcontroller based orincluded in a state machine. The disclosure may be a combination of theabove intelligence and memory and this can exist in a central processingunit or a multiple of chips including a central graphics chip. Thecomputer portion of the disclosure may also include a model viewcontroller (MVC) as shown in FIG. 8, which is also called herein a“model controller.”

Dithering

According to an embodiment of the present disclosure, dithering can beemployed to modify and improve the visual experience of the viewer.Dithering can be effected, for example, by rapid in-plane vibration of acamera or image generator 10 using a piezoelectric actuator 122, as wasdescribed previously with respect to FIGS. 11A-D. Dithering, imparted tothe displayed image content using synchronous, timed spatialdisplacement, can be a desirable solution for helping to mask oreliminate display-related artifacts, for example.

Dithering can also be used to enhance image resolution using TimedSpatial Displacement. Improved image resolution is goal, and holdspromise for future use of AR/VR glasses in various applications, such asin critical use cases such as surgery visualization. In theseapplications, for example, detail visualization of fine layers and bloodvessels can be critical to successful outcomes. Micro displays continueto mature, with pixel counts of 2 million in a single small device, withfurther improvements likely. These higher resolution displays imposesteeply increased demands on system resources, including higher powerand computation speed and complexity, for example.

The Applicant addresses this problem using dithering of displaycomponents. This solution can make higher resolution images available tousers at a discounted power cost, which in turn can provide lighter,cooler running systems.

Increased pixel resolution is obtained by using the capability to shiftimage generator 10 in-plane, that is, in one or more (x-y plane)directions parallel to the emissive display surface of image generator10, synchronously with corresponding changes in image data content. Withrespect to FIG. 11D, image generator 10 translation is in the x-y plane,orthogonal to axis A (conventionally considered the z-axis) fordithering to increase pixel resolution using synchronous, timed spatialdisplacement. Actuator 122 is configured to provide dithering usingin-plane translation of image generator 10.

As shown schematically in FIG. 13, multiplication of the image generator10 resolution is accomplished by physically shifting an array 13 ofpixels that form image generator display 10 in the x-y plane. The shiftdistance is a proper fraction of the pixel pitch. At right in FIG. 13 isrepresented a pixel 130 of array 13, shifted in the x-direction to pixelpositions 134 and 138 and shifted in the y-direction to positions 132and 136. Synchronous with the shift action is modulation of image datafor each pixel 130 at its original position and at each shifted position132, 134, 136, and 138. Thus, for example, the state of a pixel at itsposition 134 (e.g. brightness or color) can differ from its state atposition 138, according to the image data content that is provided withthe shift. With a half-pixel shift in each x- and y-direction, forexample, the effective pixel count can be increased by at least 4 times.With a half-pixel shift only along one axis, such as only along the x ory axis as shown, the effective resolution along the axis parallel to theshift can be doubled. Overall, the power cost of nano scale piezoshifting is much lower than the cost to design and implement 4 x thepixel count using a higher resolution image generator 10 element. Anarray 14 represents increased pixel resolution.

For the embodiment of FIG. 13, an image generator 10 was provided,having a 240 frames per second (fps) refresh rate. In terms of thepiezoelectric actuation provided, each pixel element can be relocated at½ the delta of the pixel element center-to-center distance in the array13. This arrangement can provide 60 fps display at 4× the resolution ofthe original image generator 10.

By way of example, an embodiment of the present disclosure employsQNP-XY Series Two-Axis, CY Piezo Nanopositioners for image generator 10dithering actuation.

A number of features allow piezoelectric dithering to provide enhancedresolution, including the following:

(1) Adjustable travel range, such as from 100 um to 600 um, for example;

(2) Long device lifetimes;

(3) Superior positioning resolution; and

(4) High stiffness and other factors.

A number of piezoelectric actuators provide the option of closed-loopfeedback that allows sub-nanometer resolution and high linearity.

Dithering for increased resolution can utilize any of a number ofmovement patterns for in-plane displacement of pixels. Patterns fordithering the pixels in the display (or the displays) for increasedresolution include transposing with rectilinear motion, curvilinearmotion, or in a translational, rotational, periodic, or non-periodicmotion, or any combination of the above. Also, the pattern could includea rectangle pattern which may increase the resolution by 4 times.Another alternative way to address the pixel movement is to have pixelsthat are approximately the same size as the non-emissive dark or “black”space between the pixels where the dithering translation of each pixelin a display is dithered to the next adjacent unused space existingbetween the pixels in the display.

For viewer comfort, a strap adjustment can be provided, allowing both aone-time fastener positioning adjustment and a flexible stretch band.

IPD Adjustment Apparatus

As noted previously, eye gaze tracking by components in headset 100enables the system logic to determine adjustment of the inter-pupildistance (IPD) of each user. Pupillary distance (PD) or interpupillarydistance (IPD) is the distance, measured in millimeters, between thecenters of the pupils of the eyes. This measurement is different fromperson to person and typically ranges from 51.0 to 74.5 mm for women and53 to 77 mm for men. IPD adjustment is beneficial and provides metricsthat can show the amount of movement that would be required for anindividual viewer. IPD sensing can be a one-time operation and canoptionally be stored for each viewer who uses the headset. Embodimentsof the present disclosure also enable headset 100 to provide continuousmonitoring of viewer features related to IPD and to update currently IPDadjustment settings before, after, or during use of the displayapparatus.

Benefits of IPD adjustment include reduced eyestrain, improved accuracyof displayed content and improved overall appearance of 3D imaging. Tomeet the need for IPD adjustment with minimum impact on weight, size,and usability of the wearable display, an embodiment of the presentdisclosure uses an IPD adjustment apparatus that is compact,lightweight, and offers a high degree of flexibility and accuracywithout significant increase in parts count.

In addition, the apparatus can provide sufficient flexure with respectto multiple axes, allowing relatively unconstrained movement for IPDadjustment and alignment as well as for other positional adjustments tooptical and sensing systems of the head-mounted display.

FIG. 14 shows a perspective view of a wearable display headset 100 thatis configured to maintain IPD adjustment suited to the individualviewer. FIG. 15 shows portions of headset 100 removed from the viewerforehead. The distance between right-eye optical system or module 42 rand left-eye optical module 42 l can be monitored and adjusted for eachindividual viewer, using the apparatus shown and described withreference to subsequent FIGS. 16-19.

FIG. 16 shows internal components of headset 100 with an external cover1002 removed. A stationary printed circuit board (PCB) 1004 is mountedto a frame 1014 of headset 100, so that stationary circuit board 1004does not move relative to frame 1014. To provide the movement of opticaland sensing components in at least the x-axis direction for IPDadjustment, a movable printed circuit board 1008, shown more clearly inFIG. 17, is coupled to a transport apparatus 1012. Each optical module42 l, 42 r can have a movable printed circuit board 1008, as showngenerally for the right-eye optical module 42 r in FIG. 16 andfollowing. Transport apparatus 1012 also conveys the optical apparatusfor image generation and presentation and, optionally, a camera 1024,shown as mounted behind a camera shroud 1022 in the FIG. 16, 17embodiment. It is important that the IPD adjustment be correct for eachuser, and also that the dual cameras remain centered over the user'spupil for the best real-world alignment.

With the arrangement thus described, stationary and movable componentsare provided the needed connection to each other for proper operationand communication and, at the same time, have the needed separation toallow adjustment and positioning of components associated withparticular vision characteristics of the viewer.

In order to better understand benefits and structural aspects of headset100 design, it is useful to identify particular features that make IPDadjustment and fine-tuning possible. FIG. 17 shows further details ofthe internal component arrangement of headset 100 according to anembodiment of the present disclosure, with stationary circuit board 1004(FIG. 16) removed. Considering the optical arrangement for right-eyeviewing, movable circuit board 1008 is configured for conveyance along apair of rails 1018. Drive energy is provided by energizing an actuator1026, which can be a small motor or other type of rotary or linearactuator. Actuator 1026 can be piezoelectric, for example. Rails 1018extend along the x-axis from a stationary center support 1020.

In order to provide ongoing connection between movable and stationarycomponents at each incremental position of the optical system, anembodiment of the present disclosure employs a flexible circuit 1030that has at least a first connector 1032 to stationary circuit board1004 (as shown in FIG. 16) and a second connector 1034 to movablecircuit board 1008 (as shown in FIG. 17).

The perspective views of FIGS. 18A, 18B, and 18C show a basic form offlexible circuit 1030 used in each left- and right-eye optical system ormodule 42 l, 42 r as a type of ribbon cable, having printed circuittraces as well as connectors, and further mounting any number ofauxiliary components for signal conditioning. As shown in FIG. 18C,flexible circuit 1030 can provide an arrangement of bends, curves, orfolds F, respectively parallel and orthogonal, that allow circuitflexure for a range of movement between stationary and movablecomponents. Using the configuration shown, two or more of the folds Fcan be mutually orthogonal to each other, for example. In addition tocircuit traces for signal routing, flexible circuit 1030 can alsoinclude one or more electronic components mounted thereon, such asdiscrete components mounted on areas of the surface for executing logiccontrol, signal conditioning, and other functions as well as forproviding or conditioning power, grounding, and drive signals, forexample. The use of flexible circuit board materials, such as polyimidefilms and similar flexible substrates for example, can help to supportsmooth, continuous, and unobstructed movement of the correspondingsubassemblies of the headset 100 with added advantages of reducedconnector count and size. Circuit substrates used for flexibleconnection can also allow high circuit densities, thus reducing spacerequirements for improved compactness.

FIG. 19 shows a perspective view of an embodiment in which only theright-eye optical module 42 r is shifted; the left-eye optical module 42l is stationary with respect to the frame. It should be noted thateither or both of the left- and right-eye optical systems can be shiftedin position along the x-axis in order to adjust inter-pupil distance.The exact functions of stationary and movable components can vary fromthose indicated herein, which are given for the sake of example only,and not by way of limitation.

In addition to supporting x-axis movement, the use of flexible circuit1030 also provides a useful mechanism that allows movement of systemcomponents along or about other axes, such as for adjustment of focus orother optical adjustment.

Adjustment of the IPD distance can be performed by the transportapparatus in a number of ways, including manual adjustment and automaticadjustment, based on identifying viewer eye position. FIG. 20 is aperspective view that shows headset optics configured for IPD adjustmentaccording to an embodiment of the present disclosure that employs, as atype of transport apparatus, an adjustment apparatus 2000 using camadjustment. Headset 100 is formed with two barrels 2010 l and 2010 r forencasing right-eye and left-eye optics. Each barrel 2010 l, 2010 r has acorresponding flexible circuit 1030 and includes a heatsink 2012. Acenter support 2020 houses the cam mechanism for controlling barrel 2010l/2010 r movement.

FIG. 21A shows a front view of the headset 100 optics of FIG. 20 with aminimum IPD setting. FIG. 21B shows a front view of the headset opticsof FIG. 20 with a maximum IPD setting. In the embodiment shown, both theleft-eye and right-eye optical systems move with respect to centersupport 2020 and its actuator and supporting stationary circuitcomponents.

FIG. 22 is an exploded view showing the relative position of a cam-basedIPD adjustment apparatus 2000 with respect to headset 100 optics. Cam 22rotates under control of an IPD actuator 2032, which can be a motor.Actuator 2032 can be energized to rotate cam 2030. This rotation setsthe relative distance of pins 2034 that couple cam 2030 to each barrel2010 l, 2010 r. An inset C shows an example of an alternate embodimentemploying a rack-and-pinion adjustment apparatus 2000, that similarlyemploys a rotational actuator (not shown). Rack-and-pinion coupling isfamiliar to those skilled in the mechanical motion art.

FIGS. 23A and 23B show a top view with cam 2030 apparatus at extreme IPDadjustment positions.

FIGS. 24A and 24B show minimum and maximum settings for IPD adjustmentapparatus 2000 for an alternate embodiment using rack-and-pinionactuation for IPD adjustment.

In one embodiment, the eye-tracking subsystem in the headset canidentify the center of a person's cornea upon startup, and thenautomatically, through the model controller and IPD actuator, move thewearable display headset to the correct IPD adjustment.

The invention has been described in detail with particular reference toa presently preferred embodiment, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the disclosure. The presently disclosed embodiments are thereforeconsidered in all respects to be illustrative and not restrictive. Thescope of the invention is indicated by any appended claims, and allchanges that come within the meaning and range of equivalents thereofare intended to be embraced therein.

The invention claimed is:
 1. A wearable display apparatus comprising aheadset that is configured for display from: (a) a left-eye opticalsystem; (b) a right-eye optical system, wherein each optical system is anear-eye catadioptric pupil-forming optical system that defines acorresponding exit pupil for a viewer along a view axis and comprises:(i) an electroluminescent image generator that is energizable to directimage-bearing light for a 2D image from an emissive surface along anoptical path; (ii) a curved reflective surface disposed along the viewaxis and partially transmissive, wherein the curved reflective surfacedefines a curved intermediate focal surface; (iii) a beam splitterdisposed along the view axis and oriented to reflect light toward thecurved reflective surface; (iv) an optical image relay that isconfigured to optically conjugate the formed 2D image at the imagegenerator with a curved aerial image formed in air at the curvedintermediate focal surface, wherein the optical image relay comprises: aprism having an input surface facing toward the emissive surface of theimage generator, an output surface facing toward the curved intermediatefocal surface, and a folding surface extending between the input andoutput surfaces and configured for folding the optical path for lightgenerated by the image generator, wherein an aperture stop for the relayis formed within the prism; wherein the relay, curved mirror, and beamsplitter are configured to form the exit pupil for viewing the generated2D image superimposed on a portion of a visible object scene, whereincombined images from both left- and right-eye optical systems form a 3Dimage for the viewer; (c) a stationary circuit board that houses a firstset of electronic components having a fixed position within the headset;(d) a movable circuit board that is mechanically coupled to a transportapparatus that is configured to impart movement to either the left-eyeor right-eye optical system and that houses a second set of electroniccomponents, wherein a flexible connection maintains signal communicationbetween the movable circuit board and the first set of electroniccomponents on the stationary circuit board; and (e) an actuator that isenergizable to move the transport apparatus away from or toward thestationary circuit board according to viewer monitoring.
 2. Theapparatus of claim 1 wherein the flexible connection is provided by asubstrate that serves as the movable circuit board.
 3. The apparatus ofclaim 1 wherein the flexible connection is provided by a ribbon cable.4. The apparatus of claim 1 further comprising one or more eye-trackingsensors that are coupled to the headset.
 5. The apparatus of claim 4wherein the one or more eye-tracking sensors comprise a camera.
 6. Theapparatus of claim 5 wherein the camera is mechanically coupled to themovable circuit board.
 7. The apparatus of claim 1 wherein the flexibleconnection is provided by a substrate that is folded about two or moreaxes that are mutually orthogonal.
 8. The apparatus of claim 7 whereinthe flexible substrate has one or more discrete components mountedthereon.
 9. The apparatus of claim 1 wherein the actuator ispiezoelectric.
 10. The apparatus of claim 1 wherein the actuator drivesa cam.
 11. The apparatus of claim 1 wherein the transport apparatus hasa rack-and-pinion coupling.
 12. A wearable display apparatus comprisinga headset that is configured for display from: (a) a left-eyecatadioptric pupil-forming optical system; (b) a right-eye catadioptricpupil-forming optical system, wherein each optical system defines acorresponding exit pupil for a viewer along a view axis and comprises:(i) an electroluminescent image generator that is energizable to directimage-bearing light for a 2D image from an emissive surface along anoptical path; (ii) a curved reflective surface disposed along the viewaxis and partially transmissive, wherein the curved reflective surfacedefines a curved intermediate focal surface; (iii) a beam splitterdisposed along the view axis and oriented to reflect light toward thecurved reflective surface; (iv) an optical image relay that isconfigured to optically conjugate the formed 2D image at the imagegenerator with a curved aerial image formed in air at the curvedintermediate focal surface, wherein the optical image relay comprises: aprism having an input surface facing toward the emissive surface of theimage generator, an output surface facing toward the curved intermediatefocal surface, and a folding surface extending between the input andoutput surfaces and configured for folding the optical path for lightgenerated by the image generator, wherein an aperture stop for the relayis formed within the prism; wherein the relay, curved mirror, and beamsplitter are configured to form the exit pupil for viewing the generated2D image superimposed on a portion of a visible object scene, whereincombined images from both left- and right-eye catadioptric pupil-formingoptical systems form a 3D image for the viewer; (c) a stationary circuitboard that houses a first set of electronic components having a fixedposition within the headset; (d) a flexible circuit board that isconfigured to provide signal communication between the stationarycircuit board and a transport apparatus, wherein the transport apparatusis coupled to an actuator that is energizable to change a separationdistance between the left- and right-eye catadioptric pupil-formingoptical systems according to one or more signals conveyed along theflexible circuit board; and (e) a sensor on the headset that isconfigured to provide a signal indicative of a pupil position of theviewer relative to the left- or right-eye catadioptric pupil-formingoptical system.
 13. The apparatus of claim 12 wherein the sensor is acamera.
 14. The apparatus of claim 12 further comprising a memory thatrecords the separation distance used for the viewer.
 15. The apparatusof claim 12 wherein the camera is configured to repeat generating thesignal indicative of pupil position of the viewer during use of thewearable display apparatus.
 16. The apparatus of claim 12 wherein theflexible circuit board has one or more discrete components mountedthereon.
 17. A method for displaying an image, the method comprising:(a) providing a wearable display apparatus comprising a headset that isconfigured for display from a left near-eye catadioptric pupil-formingoptical system and a right near-eye catadioptric pupil-forming opticalsystem, wherein each catadioptric pupil-forming optical system defines acorresponding exit pupil for the viewer along a corresponding view axisand comprises: (i) an electroluminescent image generator that isenergizable to direct image-bearing light for a 2D image from anemissive surface along an optical path; (ii) a curved reflective surfacedisposed along the view axis and partially transmissive, wherein thecurved reflective surface defines a curved intermediate focal surface;(iii) a beam splitter disposed along the view axis and oriented toreflect light toward the curved reflective surface; (iv) an opticalimage relay that is configured to optically conjugate the formed 2Dimage at the image generator with a curved aerial image formed in air atthe curved intermediate focal surface, wherein the optical image relaycomprises a prism having an input surface facing toward the emissivesurface of the image generator, an output surface facing toward thecurved intermediate focal surface, and a folding surface extendingbetween the input and output surfaces and configured for folding theoptical path for light generated by the image generator, wherein anaperture stop for the relay is formed within the prism; wherein therelay, curved mirror, and beam splitter are configured to form the exitpupil for viewing the generated 2D image superimposed on a portion of avisible object scene, wherein combined images from both left- andright-eye optical systems form a 3D image for the viewer; and (v) aplurality of sensors in signal communication with a logic processor andcoupled to the headset, wherein the logic processor is configured toacquire and store measured data relating to the viewer; (b) energizingthe electroluminescent image generator to form an image; (c) relayingthe image, through the optical relay to form the intermediate aerialimage at the focal surface of the partially transmissive curvedreflective surface that is disposed along the view axis; (d) forming theexit pupil for the viewer by reflection of the aerial image from thebeam splitter surface; (e) sensing pupil position of the viewer andgenerating a resulting signal indicative of an inter-pupil distancebetween the left- and right-eye optical systems; and (f) configuring theheadset to adjust an inter-pupil distance between respective view axesof the left near-eye catadioptric pupil-forming optical system and theright near-eye catadioptric pupil-forming optical system according tothe resulting signal by connecting a movable transport apparatus tostationary components of the headset and to an actuator using a flexiblecircuit connection.
 18. The method of claim 17 wherein the movabletransport apparatus is formed on a circuit board.
 19. The method ofclaim 17 wherein the movable transport apparatus uses a rack-and-pinioncoupling.